Adaptive Streaming Perception using Deep Reinforcement Learning

Executing computer vision models on streaming visual data, or streaming perception is an emerging problem, with applications in self-driving, embodied agents, and augmented/virtual reality. The development of such systems is largely governed by the accuracy and latency of the processing pipeline. While past work has proposed numerous approximate execution frameworks, their decision functions solely focus on optimizing latency, accuracy, or energy, etc. This results in sub-optimum decisions, affecting the overall system performance. We argue that the streaming perception systems should holistically maximize the overall system performance (i.e., considering both accuracy and latency simultaneously). To this end, we describe a new approach based on deep reinforcement learning to learn these tradeoffs at runtime for streaming perception. This tradeoff optimization is formulated as a novel deep contextual bandit problem and we design a new reward function that holistically integrates latency and accuracy into a single metric. We show that our agent can learn a competitive policy across multiple decision dimensions, which outperforms state-of-the-art policies on public datasets.Comment: 19 pages, 17 figure

Breaking Language Barriers with a LEAP: Learning Strategies for Polyglot LLMs

Large language models (LLMs) are at the forefront of transforming numerous domains globally. However, their inclusivity and effectiveness remain limited for non-Latin scripts and low-resource languages. This paper tackles the imperative challenge of enhancing the multilingual performance of LLMs, specifically focusing on Generative models. Through systematic investigation and evaluation of diverse languages using popular question-answering (QA) datasets, we present novel techniques that unlock the true potential of LLMs in a polyglot landscape. Our approach encompasses three key strategies that yield remarkable improvements in multilingual proficiency. First, by meticulously optimizing prompts tailored for polyglot LLMs, we unlock their latent capabilities, resulting in substantial performance boosts across languages. Second, we introduce a new hybrid approach that synergizes GPT generation with multilingual embeddings and achieves significant multilingual performance improvement on critical tasks like QA and retrieval. Finally, to further propel the performance of polyglot LLMs, we introduce a novel learning algorithm that dynamically selects the optimal prompt strategy, LLM model, and embeddings per query. This dynamic adaptation maximizes the efficacy of LLMs across languages, outperforming best static and random strategies. Our results show substantial advancements in multilingual understanding and generation across a diverse range of languages

Sustainable Energy Consumption Monitoring in Residential Settings

The continuous growth of energy needs and the fact that unpredictable energy demand is mostly served by unsustainable (i.e. fossil-fuel) power generators have given rise to the development of Demand Response (DR) mechanisms for flattening energy demand. Building effective DR mechanisms and user awareness on power consumption can significantly benefit from fine-grained monitoring of user consumption at the appliance level. However, installing and maintaining such a monitoring infrastructure in residential settings can be quite expensive. In this paper, we study the problem of fine-grained appliance power-consumption monitoring based on one house-level meter and few plug-level meters. We explore the trade-off between monitoring accuracy and cost, and exhaustively find the minimum subset of plug-level meters that maximize accuracy. As exhaustive search is time- and resource-consuming, we define a heuristic approach that finds the optimal set of plug-level meters without utilizing any other sets of plug-level meters. Based on experiments with real data, we found that few plug-level meters - when appropriately placed - can very accurately disaggregate the total real power consumption of a residential setting and verified the effectiveness of our heuristic approach

MEGA: Multilingual Evaluation of Generative AI

Generative AI models have shown impressive performance on many Natural Language Processing tasks such as language understanding, reasoning, and language generation. An important question being asked by the AI community today is about the capabilities and limits of these models, and it is clear that evaluating generative AI is very challenging. Most studies on generative LLMs have been restricted to English and it is unclear how capable these models are at understanding and generating text in other languages. We present the first comprehensive benchmarking of generative LLMs - MEGA, which evaluates models on standard NLP benchmarks, covering 16 NLP datasets across 70 typologically diverse languages. We compare the performance of generative LLMs including Chat-GPT and GPT-4 to State of the Art (SOTA) non-autoregressive models on these tasks to determine how well generative models perform compared to the previous generation of LLMs. We present a thorough analysis of the performance of models across languages and tasks and discuss challenges in improving the performance of generative LLMs on low-resource languages. We create a framework for evaluating generative LLMs in the multilingual setting and provide directions for future progress in the field.Comment: EMNLP 202

Personalized Energy Services: A Data-Driven Methodology towards Sustainable, Smart Energy Systems

The rapid pace of urbanization has an impact on climate change and other environmental issues. Currently, 54% of the global population lives in cities accounting for two-thirds of global energy demand. Sustainable energy generation and consumption is the top humanity’s problem for the next 50 years. Faced with rising urban population and the need to achieve energy efficiency, urban planners are focusing on sustainable, smart energy systems. This has led to the development of Smart Grids (SG) that employs intelligent monitoring, control and communication technologies to enhance efficiency, reliability and sustainability of power generation and distribution networks. While energy utilities are optimizing energy generation and distribution, consumers play a key role in sustainable energy usage. Several energy services are provided to the consumers to know households' hourly energy consumption, estimate monthly electricity cost and recommendations to reduce energy consumption. Furthermore, advanced services such as demand response, can now control and influence energy demand at the consumer-end to reduce the overall peak demand and re-shape demand profiles. The effectiveness and adoption of these services highly depend on the consumers’ awareness, their participation and engagement. Current energy services seldomly consider consumer preferences such as their daily behavior, comfort level and energy-consumption pattern. In this thesis, we investigate development of personalized energy services that strive to achieve a balance between efficient-energy consumption and user comfort.Personalization refers to tailoring energy services based on individual consumers’ characteristics, preferences and behavior. To develop effective personalized energy services a set of challenges need to be tackled. First, fine-grained data collection at user and appliance level is required (data collection challenge). Mechanisms should be devised to collect fine-grained data at various levels in a non-intrusive way with minimal sensors. Second, personalized energy services require detailed user preferences such as their thermal comfort level, appliance usage behavior and daily habits (user preference challenge). Accurate learning models to derive user preferences with minimal training and intrusion are required. Third, energy services developed needs to be easily scalable, from one household to tens and thousands of households (scalability challenge). Mechanisms should be developed to tackle the deluge of data and support distributed storage and processing. Fourth, energy services should deliver real-time feedback or recommendations so that users can promptly act upon it (real time challenge). This calls for development of distributed and low complexity algorithms. This thesis moves away from traditional SG services -- which hardly consider consumer preferences and comfort -- and proposes a novel approach to develop effective personalized energy services. The proposed energy services provide actionable feedback, raise awareness and promote energy-saving behavior among consumers. In this thesis, we follow a bottom-up data-driven methodology to develop personalized energy services at various scales -- (i) nano: individual households, (ii) micro: buildings and spaces, and (iii) macro: neighborhoods and cities. To this end, we present our approach -- physical analytics for sustainable, smart energy systems -- that combines IoT data, physical modeling and data analytics to develop intelligent, personalized energy services. Physical analytics fuses data from various Internet of Things (IoT) devices such as smart meters, smart phones and smart watches, along with physical information such as household type, demographics and occupancy to infer energy-usage patterns, user behavior and discover hidden patterns. This approach is used to learn and model user preferences and energy usage, subsequently, employed to develop personalized energy services. This thesis is organized into three parts. Part I describes how to derive fine-grained information with minimal sensors and intrusion. We present two novel algorithms viz., LocED and PEAT that derive fine-grained information from appliance and user level, respectively. This real-time information is used to raise awareness on energy-usage behavior among occupants. Part II presents personalized energy services targeted at households and buildings. We develop services that shift and/or reduce energy consumption and cost by considering individual consumers’ preferences and comfort. These energy services are aimed at providing actionable feedback to occupants towards sustainable energy usage. Part III presents energy services targeted at neighborhood and city level. These energy services aim to identify target consumers in a neighborhood based on their energy-usage pattern and preferences for various DR programs. Finally, we present data-processing architectures that investigate how to cope with the overwhelming data generated from smart meters towards design and development of sustainable, smart energy systems.This thesis advocates that the design and development of energy services should follow personalized approach with consumer preferences and comfort given paramount importance. Results show that the personalized energy services developed has significant potential to raise awareness, reduce energy consumption and improve user comfort in smart -- homes, buildings and neighborhoods.Embedded and Networked System

Efficient Power Sharing at the Edge by Building a Tangible Micro-Grid: The Texas Case

Information and Communication Technology (ICT) is now touching various aspects of our lives. The electricity grid with the help of ICT is transformed into Smart Grid (SG) which is highly efficient and responsive. It promotes twoway energy and information flow between energy distributors and consumers. Many consumers are becoming prosumers by also producing energy. The trend is to form small communities of consumers and prosumers leading to Micro-grids (MG) to manage energy locally. MGs are parts of SG that decentralize the energy flow by allocating the produced energy within the community. Energy allocation amongst them needs to solve issues viz., (i) how to balance supply/demand within micro-grids; (ii) how allocating energy to a user affects his/her community. To address these issues we propose six Energy Allocation Strategies (EASs) for MGs - ranging from simple to optimal. We maximize the usage of the energy generated by prosumers within MG. We form household-groups sharing similar characteristics to apply EASs by analyzing thoroughly energy and socioeconomic data of households. We propose four metrics to evaluate EASs. We test our EASs on the data from 443 households over a year. By prioritizing specific households, we increase the number of fully served households up to 81 compared to random sharing.Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.Embedded and Networked System

Smart Network Interface Selection for E-DTNs

We implement two energy models that accurately and comprehensively estimates the system energy cost and communication energy cost for using Bluetooth and Wi-Fi interfaces. The energy models running on a system is used to smartly pick the most energy optimal network interface so that data transfer between two end points is maximized